1. Field of the Invention
The present invention relates generally to multimedia broadcast systems and, in particular, to a method for distributing end-user selectable additional content in a multimedia broadcast stream.
2. Description of Prior Art
The advent of digital television has enabled advances in the area of user selectable content used in conjunction with the traditional broadcast distribution architecture. Besides increasing the number of channels available, content providers are able to provide a range of services far exceeding those available in analog-based distribution systems.
The Vertical blanking interval utilized to transmit data along with the video signal is described in a number of patents, including a U.S. Pat. No. 4,536,268. However, this technique does not allow user interaction.
Other patents, including U.S. Pat. No. 5,512,935, describe the distribution of digital video requiring additional equipment such as a set too box (STB) at the receiver location. Downstream content which is broadcast from a server utilizes the traditional broadcast medium, while upstream content, such as user choices, use the public telephone system co-ordinated via the STB.
U.S. Pat. No. 5,260,778 describes embedding additional controls into digital video streams, and requires an addressable controller to transmit this stream to a receiver.
MPEG-2 Systems Specification, ISO/IEC 13818-1, specifies the MPEG-2 standard for how programs consisting of audio and video elementary streams can be multiplexed together in a xe2x80x9ctransport stream.xe2x80x9d The MPEG-2 Systems Specification accommodates the inclusion of xe2x80x9cprivate dataxe2x80x9d streams into a program""s transport stream. xe2x80x9cProgram-Specific Informationxe2x80x9d carries the information regarding which elementary streams have been multiplexed in the transport stream, what type of content they carry and how they may be demultiplexed.
The standard encoding and compression of a video stream into MPEG format is well known to those skilled in the art. For completeness purposes the general procedure is summarized here.
Though several variations to the MPEG encoding procedure exist, they are all based on JPEG, a compression standard for single image compression. In JPEG, image Q is transformed into the frequency domain using a Fast Fourier Transform (FFT) resulting in FFT(Q). Since the image is discretized in the frequency domain, losses should be expected when applying the inverse Fourier Transform to obtain the original image.
Before compression, the various frequencies are bit encoded using fewer number of bits for high frequencies and greater number of bits for low frequencies. The reasoning for this is that high frequencies representing changes in the image in the detail are filtered out or can be compensated by the human eye. Hence, less information has to be dedicated to the higher frequency range. In contrast, low frequencies describe the overall image and are not filtered by the eye. The resulting encoded bitstream of FFT(Q) is a representation of the image Q with a potential small error due to the discretization. The image Q can be reconstructed by decoding the bitstream and applying the inverse Fourier Transform to FFT(Q). The encoded bitstream is then compressed using an LZ-Type compression. The compressed image of the original image Q may be denoted by JPEG(Q).
In MPEG this principle is taken further. For a given image stream IS(i), each individual image (I) I(i,j) is compared with its previous individual image I(i,jxe2x88x921) and the difference image (D) D(i,j) is computed. A difference image can be characterized using:
(i) a shift (u,v) in the image in the (x,y) direction, thus capturing a movement in the image; and
(ii) the set of pixels that have changed.
The computational complexity of the MPEG encoding procedure lies in the fact that various combinations of (u,v) have to be tried to obtain a small image D(i,j).
The D(i,j) is then compressed using a bit encoding similar to the one discussed in the JPEG compression. On the average, D(i,j) represents a small image change, the resulting compressed difference image (DC) DC(i,j) is expected to be rather small because most of the image should be the same as the previous and hence will not show up in the FFT.
Due to the fact that a sequence of DC(i,j), where (j=1, . . . , p) 100, shown in FIG. 1, is computed of each other and each DC(i,j) 100 compression can have a marginal error due to the FFT, every so often a whole compressed image I(i,j) has to be transferred to limit error compounding. Such compressed images are called anchor frames, they are created by JPEG compression, i.e., JPEG(I(i,j)), and represented as JC(i,j) 120. By default, the first image I(i,1) must be transferred as an anchor frame JC(i,1). The resulting compressed stream S(i) consists of a sequence of stream parts, each stream part defined to begin with an anchor frame JC(i,j) 120 and subsequent compressed difference images DC(i,j+1) to DC(i,j+p) 140.
Having thus described the general MPEG encoding mechanism, it is shown that the way the main image stream. IS(1) and each additional image stream IS(i) are encoded, is as independent MPEG streams S(i) 130. However, this approach may require additional bandwidth because the entire image part JC(i,j) 120 to DC(i,j+p) 140 has to be transmitted because the difference images are computed of JC(i,j) 120 and its subsequent images.
In providing for some level of customization, additional items beyond a broadcast architecture need to be supplied:
1) a means of communicating user""s preferences from the receiver back to the source of the broadcast program, and
2) a separate channel for transmission of the modified content back to the originator of the preference.
These additions add expense and complexity to the broadcast system, i.e., each user must be assigned a separate channel.
What is needed is a method for substantially reducing or eliminating above limitations, all the while retaining the cost and simplicity of a multimedia broadcast architecture would be useful.
The invention is a system architecture which, in addition to the base stream, enables the broadcast delivery of alternative media segments to a receiver, without a separate broadcast channel for these alternative segments and without a need for an upstream channel for that receiver to indicate which alternative media segment to view.
In one embodiment of the invention the content owners may choose to offer an option of a PG-rated version of a R-rated movie. Most of the movie may be rated PG, however a few segments may give it the R rating. The PG-rated version would be identical to the original except for the substitution of a few brief segments. The base stream would be the R-rated version of the movie, and the alternative segments would be those PG-rated segments which would substitute for the R-rated segments. The invention allowing the user to choose which version to view by broadcasting a combination of the R and PG rated versions without the use of additional upstream or downstream channels.
The base media stream may be compressed using any number of methods. During pauses in the instantaneous bandwidth requirements of the base stream, the compressed alternative media segments are multiplexed with the base media stream to create a combined stream. Typically, the combined stream""s instantaneous bandwidth requirement will exceed the channel""s capacity to deliver this stream. In those cases the compression of the alternative segments is modified by expanding the delivery time for the alternative segment, hence reducing the combined stream""s instantaneous bandwidth requirements. The alternative segments are expanded by an amount great enough so that the combined stream""s bandwidth requirements can be satisfied by the channel. Because the alternative segment is expanded in time, the delivery of the alternative segment must be initiated far enough in advance of its use so that it will be present at the receiver at the appropriate time.
Upon receiving the combined stream, the receiver demultiplexes the combined stream into the base stream, and alternative segments. The base stream proceeds through the receiver as usual, while the alternative segments are buffered into some storage and reconstructed into a suitable form for playback at the correct time. The user selects which, if any, of the alternative streams to view, and at the appropriate moment, the reconstructed alternative stream is merged, or replaced, with the base stream and displayed.
The number of alternative streams and their parameters, e.g., duration, quality, etc . . . , that can be encoded into the base stream in this way is determined by the instantaneous bandwidth requirements of the base stream, the channel capacity, the alternative streams parameters and the temporal locations for their display during playback of the base stream. The compression and scheduling of these segments is computed only once.
The number of alternative streams that can be displayed at the receiver is dependent on the above and also on the amount of storage and the processing power at the receiver. If the receiver does not have any storage or processing power, only the base stream is played.